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Abstract 

Grammar development environ- 
ments (GDE's) for analysis and for gener- 
ation have not yet come together. Despite 
the fact that analysis-oriented GDE's (such 
as ALEp) may include some possibility of 
sentence generation, the development tech- 
niques and kinds of resources suggested are 
apparently not those required for practi- 
cal, large-scale natural language generation 
work. Indeed, there is no use of 'standard' 
(i.e., analysis-oriented) GDE's in current 
projects/applications targetting the gener- 
ation of fluent, coherent texts. This un- 
satisfactory situation requires some analy- 
sis and explanation, which this paper at- 
tempts using as an example an extensive 
GDE for generation. The support pro- 
vided for distributed large-scale grammar 
development, multilinguality, and resource 
maintenance are discussed and contrasted 
with analysis-oriented approaches. 

1 Introduction: a problem 

Grammar development environments (GDE's) for 
analysis and for generation have not yet come to- 
gether. In fact, the mainstay of design for linguistic 
resource development environments is skewed very 
far from that necessary for generation; this is illus- 
trated well by the following remark from an eagles 
(Expert Advisory Group for Language Engineering 
Standards) report describing the current "conver- 
gence" of opinion concerning the required function- 
ality for development platforms: 

"The necessary functionality of a develop- 
ment platform is more or less agreed upon 
by grammar writers. . . . They should have 



a parser for testing the developed gram- 
mars with respect to an input string, and 
possibly also a generator to test for over- 
generation." ([EAGLES, 1996|, pi 17) 



This marginalization of the generation process nat- 
urally impacts on the kinds of development and de- 
bugging tools that are provided. For example, per- 
haps the most extensive workbench developed within 
the European Union, the Advanced Language Engi- 
neering Platform (alep: cf. (Simpkins et al., 1993)), 
while forced to adopt a so-called 'lean' formalism in 
order to achieve acceptable efficiency, nevertheless 
orients itself most closely to 'mainstream' linguis- 
tic formalisms such as HPSG and LFG. Neither of 
these formalisms have however found widespread use 
in larger-scale generation contexts. 

There also continue to be substantial projects 
whose specific goals are to build or collect linguis- 
tic resources for language engineering — including, 
for example, projects such as Acquilex, Eagles, 
TransTerm, EuroWordNet and others.^] However, 
these projects have not apparently been configured 
to provide the kinds of resources that generation 
requires. This can be seen in the virtually zero 
take-up of such 'mainstream' (i.e., analysis-oriented) 
resources in generation projects (both monolingual 
and multilingual) where the goal has been to provide 
efficient generation of realistic, useful texts. 

Thus, not only is there a lack of uptake of lin- 
guistic resources, there is also virtually no use of 
'standard' (i.e., analysis-oriented) GDE's in current 
projects/applications targetting the generation of 
fluent, coherent texts. This unsatisfactory situation 
certainly requires some analysis and explanation — 
which this paper attempts. To do this, we first 
briefly illustrate our claim that the grammar de- 
velopment environments and approaches that are 



1 Schiitz (|Schutz, 1996) provides a useful overview 



of current language engineering projects where multilin- 
guality plays a role. 



adopted in natural language generation are by and 
large disjoint to those developed in natural language 
analysis. We then show how the main property that 
effectively distinguishes successful generation gram- 
mars from analysis grammars (regardless of what 
the grammars are used for) is their orientation to 
communicative-function and that it is precisely this 
property that plays a crucial role in creating power- 
ful and efficient grammar development environments 
that are suited to the generation task. 

We illustrate this relationship between resource 
organization and development tools by focus- 
ing on techniques for developing and maintain- 
ing large-scale linguistic resources (mostly grammar 
and semantics-grammar mappings), for distributed 
grammar development, and for supporting multilin- 
guality that have developed for generation work. A 
direct question raised by the paper is then the ex- 
tent to which the techniques discussed could also be 



relevant and applicable to analysis-oriented develop- 
ment environments. 

2 The lack of use of analysis-based 
GDE's for generation 

There is clearly a partially 'sociological' explana- 
tion to the lack of exchange between approaches 
in analysis and generation: the groups working on 
analysis and text generation are by and large dis- 
joint and the questions and issues thus central in 
these groups are also at best only partially overlap- 
ping. This is not, however, sufficient. Most input 



to analysis-oriented work (e.g., (Pulman, 1991)) has 
attempted to achieve a workable level of generality 
and formal well-foundedness that would guarantee 
the widespread applicability and re- usability of their 
results. If this were sufficient and had been success- 
ful, one could expect generation developers to have 
availed themselves of these results. But uptake for 
generation continues to be restricted to those work- 
ing in the analysis-oriented tradition, mostly in the 
pursuit of 'bi-directional' sentence generation on the 
basis of resources developed primarily for analysis. 
'Core' text generation activities remain untouched. 

One, more contentful, reason for this is that 
the particular requirements of generation favour an 
organization of linguistic resources that has itself 
proved supportive of powerful development and gen- 
eration environments. To clarify the needs of gen- 
eration and the relation to the GDE's adopted, we 
can cross-classify approaches adopted for generation 
according to the kind of generation targetted. This 
largely corresponds to the size of linguistic unit gen- 
erated. Thus we can usefully distinguish generation 



of single phrases, generation of single sentence or 
utterance generation (such as might also still occur 
in MT most typically or in utterance generation in 
dialogue systems) , generation of connected texts of 
a single selected text type, and generation of con- 
nected texts of a variety of text types (e.g., showing 
variation for levels of user expertise, etc.). These are 
distinguished precisely because it is well known from 
generation work that different issues play a role for 
these differing functionalities. 

Three generation 'environments' cover the ma- 
jority of projects concerned with text generation 
where generation for some practical purpose (s) is 
the main aim, not the development of some par- 
ticular linguistic treatment or pure research into 
problems of ge neration or NLP generally. These 
are Elhadad's ( Elhadad, 1990 ) 'Functional Unifi- 
cation Formalism' (fuf), the KPML/P e nman sys- 
tems ( Mann and Matthicsscn, 1985| ; Bateman 



1997), and approaches within the Meaning- Text 
Model (cf. flMel'cuk and Zholkovskij, 1970| )) as used 
in the CoGenTex-family of generators. Here re- 
sources appropriate for real generation are accord- 
ingly understood as broad coverage (with respect 
to a target application or set of applications) lin- 
guistic descriptions of languages that provide map- 
pings from enriched semantic specifications (includ- 
ing details of communicative effects and textual or- 
ganization) to corresponding surface strings in close 
to real-time. In addition, there are many systems 
that adopt in contrast a template-based approach 
to generation — now often combined with full gen- 
eration in so-called 'hybrid' frameworks. While, 
finally there is a very small number of serious, 
large-scale and/or practical projects where analysis- 
derived grammatical resources are adopted. This 
distribution is summarized in Table |l|. Importantly, 
it is only for the approaches in the final righthand 
column that standard analysis-based GDE's appear 
to be preferred or even applicable. 2 

3 Communicative function: a 
common thread in generation 
resources 

It is well known in natural language generation 
(NLG) that functional information concerning the 
communicative intent of some utterance provides a 
convenient and necessary organization for generator 



decisions (cf. ([McDonald, 1980| ; [Appelt, 1985| ; |McK 



references to individual systems see the Web 
or a det ailed current state of th e art such as Zock and 
Adorni f|Zock and Adorni, 1996| ) or Bateman (Bateman 
|to appear^ 
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Table 1: Distribution of generation systems by task and approach 



eown, 1985)). Different approaches focus the role 
of communcative functions to a greater or less de- 
gree. Some subordinate it entirely to structure, some 
attempt to combine structure and function felici- 
tously, others place communicative function clearly 
in the foreground. Among these latter, approaches 
based on systemic-functional linguistics (SFL) have 
found the widest application. Both the fuf and 
KPML/Penman environments draw heavily on SFL. 
This is to emphasize the role of the paradigmatic 
organization of resources in contrast to their syn- 
tagmatic, structural organization. It turns out that 
it is this crucial distinction that provides the clean- 
est account of the difference between a GDE such as 
ALEP and one such as KPML. 

Viewed formally, a paradigmatic description of 
grammar such as that of SFL attempts to place 
as much of the work of the description in the type 
lattice constituting the grammar. The role of con- 
straints over possible feature structures is minimal. 
Moreover, the distinctions represented in the type 
lattice subsume all kinds of grammatical variation — 
including variations that in, for example, an HPSG- 
style account might be considered as examples of the 
application of lexical rules. Diathesis alternations 
are one clear example; differing focusing construc- 
tions are another. These are all folded into the type 
lattice. Generation with such a resource is then re- 
duced to traversing the type lattice, generally from 



least-specific to most-specific types, collecting con- 
straints on structure. A grammatical unit is then 
exhaustively described by the complete list of types 
selected during a traversal: this is called a selection 
expression. Additional mechanisms (in particular, 
the 'choosers') serve to enforce determinism: that 
is, rather than collect parallel compatible partial se- 
lection expressions, deterministic generation is en- 
forced by appealing to semantic or lexical informa- 
tion as and when required. This approach, which is 
theoretically less than ideal, in fact supports quite 
efficient generation. It can be equated with the use 
of 'lean' formalisms in analysis-oriented GDE's. 

This paradigmatic design sketched here has 
proved to have significant consequences for the de- 
sign of appropriate development environments. The 
properties of these development environments are 
also directly inferable from the properties of the lin- 
guistic descriptions they are to support. Among the 
results are: 

• a much improved mode of resource debugging, 

• a powerful treatment of multilinguality in lin- 
guistic resources, 

• and strong support for distributed large-scale 
grammar development. 

We will briefly note these features and then present 
some derivative functionalities that also represent 
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Figure 1: Accessing points in the grammatical type lattice from a generated structure 



differences between analysis and generated oriented 
GDE's. For the functional approaches, our con- 
crete descriptions will be based on kpml: fuf is 
not explicitly multilingual and has as yet few vi- 
sualization tools for resource development (limited, 
for example, to basic graphs of the type lattice). 
Kpml is more similar in its stage of development 
to, for example, ALEP, in that it offers a range of vi- 
sualisation techniques for both the static resources 
and their dynamic use during generation, as well as 
support methods for resource construction, includ- 
ing versioning, resource merging, and distinct kinds 
of modularity for distributed development. Fuf is 
still mostly concerned with the underlying engine 
for generation and represents a programming envi- 
ronment analogous to CUF or tdl. 

Beyond interactive tracing 

Experiences with debugging and maintaining 
large generation grammars lead to the conclusion 
that 'tracing' or 'stepping' during execution of the 
resources is usually not a useful way to proceed. This 
was the favored (i.e., only) mode of interaction with, 
for example, the Penman system in the 80s. This has 
been refined subsequently, both in Penman and in 
KPML and fuf, so that particular tracing can occur, 



interactively if required, only when selected linguis- 
tic objects (e.g., particular disjunctions, particular 
types of 'knowledge base' access, etc.) are touched 
during generation or when particular events in the 
generation process occurred. However, although al- 
ways necessary as a last resort and for novices, this 
mode of debugging has now in kpml given way com- 
pletely to 'result focusing'. Here the generated result 
(which can be partial in cases where the resources 
fail to produce a final generated string) serves as a 
point of entry to all decisions taken during gener- 
ation. This can also be mediated by the syntactic 
structure generated. 

This is an effective means of locating resource 
problems since, with the very conservative 'formal- 
ism' supported (see above), there are only two pos- 
sible sources of generation errors: first, when the 
linguistic resources defined cover the desired gen- 
eration result but an incorrect grammatical feature 
is selected (due to incorrect semantic mappings, or 
to wrongly constrained grammatical selections else- 
where); and second, when the linguistic resources 
do not cover the desired result. This means that the 
debugging task always consists of locating where in 
the feature selections made during generation — i.e., 
in the selection expressions for the relevant gram- 



matical units — an inappropriate selection occurred. 

The selection expression list is accessed from the 
user interface by clicking on any constituent, either 
from the generated string directly or from a graphi- 
cal representation of the syntactic structure. The list 
itself can be viewed in three ways: (i) as a simple list 
functioning as a menu, (ii) as a graphical representa- 
tion of the type lattice (always a selected subregion 
of the lattice as a whole) with the selected features 
highlighted, and (iii) as a animated graphical trace 
of the 'traversal' of the type lattice during genera- 
tion. In addition, all the structural details of a gen- 
erated string are controlled by syntactic constraints 
that have single determinate positions in the type 
lattice. It is therefore also possible to directly in- 
terrogate the generated string to ask where particu- 
lar structural features of the string were introduced. 
This is a more focused way of selecting particular 
points in the type lattice as a whole for inspection. 

Figure ^ shows a screenshot during this latter kind 
of user activity. The user is attempting to find out 
what where the lexical constraints responsible for 
the selection of the noun "TIME" in the phrase "At 
the same TIME" were activated. Selecting to see the 
lexical class constraints imposed on this constituent 
(THING#3 in the structure top-right) gives a listing 
of applied constraints (lower-right). This indicates 
which lexical constraints were applicable (e.g., NOUN, 
COMMON-NOUN, etc.) and where in the type lattice 
these constraints were introduced (e.g., at the dis- 
junction named HEAD-SUBSTITUTION, etc.). Click- 
ing on the disjunction name brings up a graphical 
view of the disjunction with the associated struc- 
tural constraints (upper- left). The feature selected 
from a disjunction is highlighted in a different color 
(or shade of grey: lexical-thing). The 'paradig- 
matic context' of the disjunction (i.e., where in the 
type lattice it is situated) is given to the left of the 
disjunction proper: this is a boolean expression over 
types presented in standard systemic notation. 

Several directions are then open to the user. The 
user can either follow the decisions made in the type 
lattice to the left (less specific) or to the right (more 
specific): navigating in either case a selected sub- 
graph of the type lattice. Alternatively, the user can 
inspect the semantic decisions that were responsible 
for the particular selection of grammatical feature in 
a disjunction. This 'upward' move is also supported 
graphically. The particular decisions made and their 
paths through semantic choice experts ('choosers') 
associated with each (grammatical) disjunction are 
shown highlighted. Since all objects presented to the 
user are mouse-sensitive, navigation and inspection 
proceeds by direct manipulation. All objects pre- 



sented can be edited (either in situ or within auto- 
matically linked Emacs buffers). Any such changes 
are accumulated to define a patch version of the 
loaded resources; the user can subsequently create 
a distinct patch for the resources, or elect to accept 
the patches in the resource set. Generation itself is 
fast (due to a simple algorithm: see above), and so 
creating a new 'result string' for further debugging 
in the face of changes made is the quickest and most 
convenient way to conduct further tests. This elim- 
inates any need for backtracking at the user devel- 
opment level. It is possible to examine contrastively 
the use of resources across distinct generation cycles. 

One useful way of viewing this kind of activity is 
by contrast to the state of affairs when debugging 
programs. Kpml maintains the linguistic structure 
as an explicit record of the process of generation. All 
of the decisions that were made during generation 
are accessible via the traces they left in the generated 
structure. Such information is typically not available 
when debugging a computer program since when the 
execution stack has been unwound intermediate re- 
sults have been lost. If certain intermediate results 
must consequently be re-examined, it is necessary 
to introduce tracing at appropriate points — a pro- 
cedure that can now usually be avoided resulting in 
significantly faster cycles of debugging/testing. 

Multilingual representations 

The use of multilingual system networks has been 
motivated by, for example, Bateman, Matthiessen, 
Nanri and Zeng (Bateman et al., 1991). Kpml 
provides support for such resources, including con- 
trastive graphical displays of the type lattices for dis- 
tinct languages. In addition, it is possible to merge 
automatically monolingual or multilingual resource 
definitions and to separate them out again as re- 
quired. Importing segments of a type lattice for one 
language to form a segment for a distinct language 
is also supported. This has shown that it is not nec- 
essary to maintain a simple division between, for ex- 
ample, 'core' grammar and variations. Indeed, such 
a division is wasteful since language pairs differ in 
the areas they share. The support for this multi- 
linguality is organized entirely around the paradig- 
matic type lattice. The support tools provided for 
manipulating such language-conditionalized lattices 
in kpml appear to significantly reduce the devel- 
opment time for generation resources for new lan- 
guages. A black-and-white representation of a con- 
trastive view based on the Eagles morphology rec- 
ommendations is shown in Figure |[ The graph em- 
phasizes areas held in common and explicitly labels 
parts of the lattice that are restricted in their lan- 
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Figure 2: Views on a multilingual resource 



guage applicability. 

The possibilities supported for working multilin- 
gually (e.g., inheritance, merging resources) rely en- 
tirely on the relative multilingual applicability of the 
paradigmatic organization of the grammar. It ap- 
pears a fact of multilingual description that paradig- 
matic functional organizations are more likely to 
show substantial similarities across languages than 
are the syntagmatic structural descriptions. In an 
overview of resource definitions across 6 languages, 
it was found that the multilingual description only 
contains 32% of the number of objects that would 
be need if the 6 grammars were represented sepa- 
rately. Significant degrees of overlap have also been 
reported whenever a description of one language 
has been attempted on the basis of another (cf., 
e.g., (|Alshawi et al., 1992); |Rayner et al., 1996|)). 



The paradigmatic basis simply extends the range 
of similarities that can be represented and provides 
the formal basis for providing computational tools 
that support the user when constructing language 
descriptions 'contrastively'. 

Distributed large-scale grammar 
development 

The paradigmatic organization of a large-scale 
grammar shows a further formal property that is 
utilized throughout the KPML GDE. Early work on 
systemic descriptions of English noted that emer- 
gence of 'functional regions': i.e., areas of the gram- 
mar overall that are concerned with particular areas 



of meaning. As Halliday notes: 

"These [functional] components are re- 
flected in the lexicogrammatical system 
in the form of discrete networks of op- 
tions. Each ... is characterized by strong 
internal but weak external constraints: for 
example, any choice made in transitivity 
[clause complementation] has a significant 
effect on other choices within the transi- 
tivity systems, but has very little effect 
on choices within the mood [speech act 
types] or theme [information structuring] 
systems." flHalliday, 197S|, pi 13). 



This organization was first used computationally in 
the development of the Nigel grammar of English 
within the Penman project. Nigel was probably the 
first large-scale computational grammar whose pre- 
cise authorship is difficult to ascertain because of the 
number of different linguists who have contributed 
to it at different times and locations. 

The basis for this successful example of dis- 
tributed grammar development is the organization 
of the overall type lattice of the grammar into mod- 
ular functional regions. This has now been taken 
as a strong design principle within kpml where all 
user access to the large type lattices making up a 
grammar is made through specific functional regions: 
for example, asking to graph the lattice will by de- 
fault only present information within a single re- 
gion (with special pointers out of the region to indi- 
cate broader connectivity). This is the paradigmatic 
equivalent of maintaining a structural grammar in 
modules related by particular syntactic forms. How- 
ever, whereas the latter information is not strongly 
organizing for work on a generation grammar, the 
former is: work on a generation resource typically 
proceeds by expanding a selected area of expressive 
potential — i.e., the ability of the grammar to express 
some particular set of semantic requirements. This 
can include a range of grammatical forms and is 
best modularized along the paradigmatic dimension 
rather than the syntagmatic. 

The relative strength of intra-region connections 
in contrast to extra-region connections has provided 
a solid basis for distributed grammar development. 
Developers typically announce that they are inter- 
ested in the expressive potential of some functional 
region. This both calls for others interested in the 
same functional region to exchange results coopera- 
tively and warns generally that a functional region 
may be subject to imminent change. When a revised 
version of the region is available it replaces the pre- 
vious version used. Integration of the new region 




Figure 3: Functional region connectivity for English (extract) 



is facilitated by a range of visualization tools and 
connectivity checks: the hnal test of acceptability 
is that all test suites (see next subsection) generate 
the same results as with the previous region version 
and that a new test suite is provided that demon- 
strates the increased or revised functionality of the 
new region. 

Regions are dehncd across languages: the current 
multilingual resources released with kpml include 
around 60 regions. A partial region connectivity 
graph for the English grammar is shown in Figure |^. 
This graph also serves as a 'menu' for accessing fur- 
ther graphical views of the type lattice as well as 
selections from test suites illustrating use of the re- 
sources contained within a region. Dependencies be- 
tween regions are thus clearly indicated. 

Integrated test suites 

Sets of linguistic resources for generation are typ- 
ically provided with test suites: such test suites con- 
sist minimally of a semantic specification and the 
string that should result when generating. In KPML 
these are indexed according to the grammatical fea- 
tures that arc selected during their generation. This 
permits examples of the use and consequences of any 
feature from the type lattice to be presented during 
debugging. This is one particularly effective way not 
only of checking the status of resources but also for 
documenting the resources. The complete genera- 
tion history of examples can be examined in exactly 
the same way as newly generated strings. An inter- 



esting line of development underway is to investigate 
correspondences between the paradigmatic features 
describing features in a KPML-example set and those 
features used in the TSNLP initiative. 

4 Discussion 

The basic premises of a generation-oriented GDE 
such as KPML differ in certain respects to those those 
of an analysis-oriented GDE such as ALEP. This also 
stretches to the style of interaction with the system. 
For example, interaction with the Kpml GDE is, 
as with Smalltalk and ALEP, object-oriented but, in 
contrast to ALEP, the objects to which a user has 
access are strongly restricted to just those linguistic 
constructs that are relevant for generation. This sep- 
arates development environment details from the re- 
sources that are being developed. This is, of course, 
both possible and desirable because kpml is not in- 
tended to be tailored for particular types of resource 
by the user: the theoretical orientation is fixed. 

The benefits of this approach seem to far outweigh 
the apparent limitations. First, the visualizations 
provided are exactly tailored to the details of the 
linguistic objects supported and their use in genera- 
tion. Thus resource sets, networks, systems (disjunc- 
tions), semantic choice experts, dynamic traversal of 
the network, syntactic structures, etc. all have their 
own distinctive graphical representations: this es- 
tablishes a clear modularity in the conception of the 
user that is easily obscured when a single more gen- 



eration representation style (e.g., feature structure 
presented in a feature structure editor) is used for a 
wide range of information. This clarifies the differ- 
ence in information modules and thus helps devel- 
opment. It is then also possible to 'fold' generation 
decisions into the visualizations in a natural way: 
thus supporting the 'result focusing' mode of devel- 
opment described above. Thus, whenever resources 
are inspected, their use during selected cycles of gen- 
eration is also displayed by highlighting or annotat- 
ing the appropriate objects shown. 

This also influences the kind of user for which the 
GDE is appropriate. The central areas in generation 
are still primarily functional and pragmatic rather 
than structural and syntactic. It is less common that 
linguists and developers concerned with pragmatics 
and text linguistics are fully comfortable with con- 
straint logic programming. The dedicated graphical 
presentation of linguistic objects provided in KPML 
appears to provide a more generally accessible tool 
for constructing linguistic descriptions. Grammar 
components have been written using KPML by com- 
puter scientists without training in computational 
linguistics, by functional text linguists, by transla- 
tors and technical writers, as well as by computa- 
tional and systemic-functional linguists. 

Finally, however, the well understood relationship 
between systemic-functional style descriptions and, 
for example, typed feature representations provides 
a bridge from the less formal, more functional style 
of description back to the kind of representations 
found in NLA-oricnted GDE's. It is therefore to 
be expected that a broader range of linguistic input 
and development work will be encouraged to feed 
into large-scale resource development than would be 
possible if the kind of development were limited to 
that practised for purposes of analysis. 
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